Relational Stacked Denoising Autoencoder for Tag Recommendation
نویسندگان
چکیده
Tag recommendation has become one of the most important ways of organizing and indexing online resources like articles, movies, and music. Since tagging information is usually very sparse, effective learning of the content representation for these resources is crucial to accurate tag recommendation. Recently, models proposed for tag recommendation, such as collaborative topic regression and its variants, have demonstrated promising accuracy. However, a limitation of these models is that, by using topic models like latent Dirichlet allocation as the key component, the learned representation may not be compact and effective enough. Moreover, since relational data exist as an auxiliary data source in many applications, it is desirable to incorporate such data into tag recommendation models. In this paper, we start with a deep learning model called stacked denoising autoencoder (SDAE) in an attempt to learn more effective content representation. We propose a probabilistic formulation for SDAE and then extend it to a relational SDAE (RSDAE) model. RSDAE jointly performs deep representation learning and relational learning in a principled way under a probabilistic framework. Experiments conducted on three real datasets show that both learning more effective representation and learning from relational data are beneficial steps to take to advance the state of the art. Introduction Due to the abundance of online resources like articles, movies, and music, tagging systems (Yu et al. 2014) have become increasingly important for organizing and indexing them. For example, CiteULike1 uses tags to help categorize millions of articles online and Flickr2 allows users to use tags to organize their photos. However, it is often not easy to compose a set of words appropriate for the resources. Besides, the large variety in phrasing styles of the users can potentially make the tagging information inconsistent and idiosyncratic. With such technical challenges, research in tag recommendation (TR) (Gupta et al. 2010; Wang et al. 2012) has gained in popularity over the past few years. An accurate tag recommendation system not only can save the pain of users searching for candidate tags on Copyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. http://www.citeulike.org http://www.flickr.com the tip of their tongues, but can also make the tags used more consistent. Consequently, both the user experience and recommendation accuracy can be improved dramatically. Tag recommendation methods can roughly be categorized into three classes (Wang et al. 2012): content-based methods, co-occurrence based methods, and hybrid methods. Content-based methods (Chen et al. 2008; 2010; Shen and Fan 2010) utilize only the content information (e.g., abstracts of articles, image pixels, and music content) for tag recommendation. Co-occurrence based methods (Garg and Weber 2008; Weinberger, Slaney, and van Zwol 2008; Rendle and Schmidt-Thieme 2010) are similar to collaborative filtering (CF) methods (Li and Yeung 2011). The co-occurrence of tags among items, usually represented as an tag-item matrix, is used for tagging. The third class of methods (Wu et al. 2009; Wang and Blei 2011; Yang, Zhang, and Wang 2013; Zhao et al. 2013; Bao, Fang, and Zhang 2014; Chen et al. 2014), also the most popular and effective ones, consists of hybrid methods. They make use of both tagging (co-occurrence) information (the tag-item matrix) and item content information for recommendation. In hybrid methods, learning of item representations (also called item latent factors in some models) is crucial for the recommendation accuracy especially when the tag-item matrix is extremely sparse. Recently, models like collaborative topic regression (CTR) (Wang and Blei 2011) and its variants (Purushotham, Liu, and Kuo 2012; Wang, Chen, and Li 2013) have been proposed and adapted for tag recommendation to achieve promising performance. These models use latent Dirichlet allocation (LDA) (Blei, Ng, and Jordan 2003) as the key component for learning item representations and use probabilistic matrix factorization (PMF) (Salakhutdinov and Mnih 2007) to process the co-occurrence matrix (tag-item matrix). However, when using LDA, the resulting item representations tend to be quite sparse. Consequently, more dimensions may be needed for the representations to be effective. Unfortunately PMF with the low-rank assumption usually works with quite a small number of latent dimensions, which is not in line with the nature of LDA (or CTR). On the other hand, deep learning models like stacked denoising autoencoder (SDAE) (Vincent et Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
منابع مشابه
Decoding Stacked Denoising Autoencoders
Data representation in a stacked denoising autoencoder is investigated. Decoding is a simple technique for translating a stacked denoising autoencoder into a composition of denoising autoencoders in the ground space. In the infinitesimal limit, a composition of denoising autoencoders is reduced to a continuous denoising autoencoder, which is rich in analytic properties and geometric interpretat...
متن کاملSMS Spam Filtering Using Probabilistic Topic Modelling and Stacked Denoising Autoencoder
In This paper we present a novel approach to spam filtering and demonstrate its applicability with respect to SMS messages. Our approach requires minimum features engineering and a small set of labelled data samples. Features are extracted using topic modelling based on latent Dirichlet allocation, and then a comprehensive data model is created using a Stacked Denoising Autoencoder (SDA). Topic...
متن کاملRemote Sensing Image Classification Based on Stacked Denoising Autoencoder
Focused on the issue that conventional remote sensing image classification methods have run into the bottlenecks in accuracy, a new remote sensing image classification method inspired by deep learning is proposed, which is based on Stacked Denoising Autoencoder. First, the deep network model is built through the stacked layers of Denoising Autoencoder. Then, with noised input, the unsupervised ...
متن کاملMultimodal Stacked Denoising Autoencoders
We propose a Multimodal Stacked Denoising Autoencoder for learning a joint model of data that consists of multiple modalities. The model is used to extract a joint representation that fuses modalities together. We have found that this representation is useful for classification tasks. Our model is made up of layers of denoising autoencoders which are trained locally to denoise corrupted version...
متن کاملTraining Stacked Denoising Autoencoders for Representation Learning
We implement stacked denoising autoencoders, a class of neural networks that are capable of learning powerful representations of high dimensional data. We describe stochastic gradient descent for unsupervised training of autoencoders, as well as a novel genetic algorithm based approach that makes use of gradient information. We analyze the performance of both optimization algorithms and also th...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015